Implement very minimalistic support for go workspaces #1250

HakanSunay · 2022-05-09T16:30:58Z

With this change, gazelle can invoke update-repos by processing go.work files.

Most of the common logic in module.go and work.go has been extracted into utils.go.
The main difference between the module and workspace mode lies in the action for fetching module sums.
For modules, only the missing sums are fetched through go mod download, but for workspaces the whole process is delegated to go mod download. This is due to the fact that go list does not show module sums yet. An upstream proposal was filed for the same, which can be seen in golang/go#52792.

Given the fact the standard go tooling in 1.18 can work with go.work files, not many changes were actually necessary.
A smoke test was also added to update_import_test.go to verify that the latest version of govmomi and rest is always picked.

How to invoke:

gazelle update-repos -from_file=go.work -to_macro=deps.bzl%go_deps -prune=True

What type of PR is this?
Feature

What package or component does this PR mostly affect?
language/go

What does this PR do? Why is it needed?
Implements minimalistic support for GoLang workspaces. It would help folks with multiple go.mod files in the monorepo.

Which issues(s) does this PR fix?

Fixes #1232

Other notes for review

Please, also do comment if we should be invoking go work sync as part of this workflow, which will sync all the child modules. Currently, this has not been implemented for the sake of not introducing any side effects to the execution of gazelle with regards to go.mod file changes.

I am open to testing suggestions and other possible improvements, that I might have missed.

With this change gazelle can invoke `update-repos` by processing `go.work` files. The basis of the change is a plain copy-paste of `update.go` into `work.go`. However, since `go list` does not show module sums, the whole process is delegated to `go mod download`. golang/go#52792 was created to track the above. The standard go tooling in 1.18 can work with `go.work` files so not many changes were actually necessary. A smoke test is also added to `update_import_test.go` to verify that the latest version of govmomi and rest is always picked. How to invoke: $ gazelle update-repos -from_file=go.work -to_macro=deps.bzl%go_deps -prune=True

sluongng · 2022-05-13T07:46:31Z

This approach makes sense to me, though I wonder if we should mandate a go work sync call prior to generating things?

Could you please make sure this PR pass CI? It would help reviewing this a lot easier.

HakanSunay · 2022-05-13T14:43:23Z

This approach makes sense to me, though I wonder if we should mandate a go work sync call prior to generating things?

Could you please make sure this PR pass CI? It would help reviewing this a lot easier.

CI is green now.
I also thought about running go work sync and was wondering if gazelle executions should be causing any side effects to the submodules (their go.mod and go.sum files).

sluongng

I have some small nits, but overall this is looking real good.

There are obvious code duplication between work.go and modules.go that I wish we could apply some DRY... but perhaps YAGNI.

language/go/work.go

sluongng · 2022-05-13T15:07:17Z

This approach makes sense to me, though I wonder if we should mandate a go work sync call prior to generating things?
Could you please make sure this PR pass CI? It would help reviewing this a lot easier.

CI is green now. I also thought about running go work sync and was wondering if gazelle executions should be causing any side effects to the submodules (their go.mod and go.sum files).

fair point.

May be add some documentation on this feature, then reminds people to run go work sync in the doc?

…nt code repetition

HakanSunay · 2022-05-13T16:46:01Z

I have some small nits, but overall this is looking real good.

There are obvious code duplication between work.go and modules.go that I wish we could apply some DRY... but perhaps YAGNI.

You are absolutely right, I think we should do it properly from the start (DRY). I have attempted to provide a more unified approach without that much code repetition for both modes, you can have a look.

sluongng

Small nits, but the rest LGTM.

Could you please update existing doc to add Go Workspace support

Once thats done, just remove draft status for other maintainers to review.

language/go/work.go

language/go/modules.go

sluongng · 2022-06-01T07:32:53Z

@linzhp @achew22 I was able to apply this patch in my repo and have things working as expected sluongng/nogo-analyzer@4769fa6

Could you guys take a look and consider merging this?

tomqwpl · 2022-06-10T18:16:06Z

This PR explains some things I am confused about with gazelle.
Bazel is supposed to be a tool for mono repos right? In a mono repo, I'm going to have lots of directories, each having a go.mod file. So far I've been confused by how I set this up, it looks like I have to set up targets in each of the modules to run the update-repos separately for each module, where running plain gazelle can be done over the whole thing from a BUILD.bazel file at the root.

So just for clarification, I have:

WORKSPACE
BUILD.bazel
module1\
   go.mod
   BUILD.bazel
module2
  go.mod
  BUILD.bazel

The BUILD.bazel file at the root has

# gazelle:prefix example.com/example
gazelle(name = "gazelle")
```

That appears to take care of generating the BUILD.bazel files for all the modules and their subdirectories. The bit I'm confused about is that I had naively tried adding

gazelle(
name = "gazelle-update-repos",
args = [
"-from_file=go.mod",
"-to_macro=deps.bzl%go_dependencies",
"-prune",
],
command = "update-repos",
)

to the same root BUILD.bazel file as well, but it appears that I have to put that individually into the BUILD.bazel file next to each go.mod file. I'm completely new to bazel, so maybe this shouldn't have been, but it felt like I should have to run that update-repos once across the whole monorepo.

Hopefully this PR helps that?

sluongng · 2022-06-11T08:06:00Z

@tomqwpl I think you are conflating some definitions here. This PR provide support for go.work file, which is Go 1.18's new workspace feature.

If you have a hard time learning Bazel/Gazelle, I would suggest using Bazel's Community Slack or StackOverFlow to ask questions.

linzhp

Tested in Uber's repo, this doesn't break existing gazelle update-repo -from_file=go.mod.

HakanSunay added 3 commits May 13, 2022 16:44

Add forgotten change in BUILD.bazel for the go package

100ee0d

Regenerate go_repository_tools_srcs.bzl to reflect work.go

aed34af

Hopefully the last file to update before getting a successful run

5b12c58

sluongng reviewed May 13, 2022

View reviewed changes

language/go/work.go Outdated Show resolved Hide resolved

language/go/work.go Outdated Show resolved Hide resolved

Extract common functionality between and into a new location to preve…

f17add5

…nt code repetition

sluongng reviewed May 13, 2022

View reviewed changes

language/go/work.go Outdated Show resolved Hide resolved

language/go/modules.go Outdated Show resolved Hide resolved

HakanSunay added 2 commits May 13, 2022 20:38

Simpliy return expressions

d8c18ce

Update general README with information about

ebe0e53

HakanSunay changed the title ~~Implement very minimalistic support go workspaces~~ Implement very minimalistic support for go workspaces May 16, 2022

HakanSunay marked this pull request as ready for review May 16, 2022 17:15

linzhp approved these changes Jun 12, 2022

View reviewed changes

linzhp merged commit 0e810ae into bazel-contrib:master Jun 12, 2022

lstanden mentioned this pull request Jan 16, 2024

Support Golang Workspaces in Gazelle #1720

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement very minimalistic support for go workspaces #1250

Implement very minimalistic support for go workspaces #1250

HakanSunay commented May 9, 2022 •

edited

Loading

sluongng commented May 13, 2022 •

edited

Loading

HakanSunay commented May 13, 2022

sluongng left a comment

sluongng commented May 13, 2022

HakanSunay commented May 13, 2022

sluongng left a comment

sluongng commented Jun 1, 2022

tomqwpl commented Jun 10, 2022

sluongng commented Jun 11, 2022

linzhp left a comment

Implement very minimalistic support for go workspaces #1250

Implement very minimalistic support for go workspaces #1250

Conversation

HakanSunay commented May 9, 2022 • edited Loading

sluongng commented May 13, 2022 • edited Loading

HakanSunay commented May 13, 2022

sluongng left a comment

Choose a reason for hiding this comment

sluongng commented May 13, 2022

HakanSunay commented May 13, 2022

sluongng left a comment

Choose a reason for hiding this comment

sluongng commented Jun 1, 2022

tomqwpl commented Jun 10, 2022

sluongng commented Jun 11, 2022

linzhp left a comment

Choose a reason for hiding this comment

HakanSunay commented May 9, 2022 •

edited

Loading

sluongng commented May 13, 2022 •

edited

Loading